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Abstract — A  nonlinear  generalization  of  the  Adaptive  Model 
Theory,  nAMT,  is  compared  with  human  open-loop  tracking 
data  across  the  same  range  of  conditions.  The  resulting 
simulations  produced  effects  that  mirrored  the  closed-  and 
open-loop  characteristics  of  the  experimental  response 
trajectories.  This  supports  the  use  of  an  internal  feedback  loop 
for  the  inversion  of  external  systems  in  the  nAMT  model.  Other 
control-systems  models  (both  AMT  and  feedback-error 
learning)  were  unable  to  reproduce  the  observed  disparity 
between  closed-  and  open-loop  results  without  fundamental 
modification.  A  low  internal  feedback  loop-gain,  incorporating 
a  substantial  derivative  component,  caused  this  effect.  This  low 
gain  produced  acceptable  performance  due  to  the  relatively  low 
target  bandwidth  used  in  the  study,  allowing  the  feedback 
control  component  to  function.  Maintenance  of  the  loop-gain  at 
the  lowest  possible  levels  is  thought  to  maximize  the  internal 
stability  of  the  inverse.  The  simulation  work  confirmed  that  the 
nAMT  model  is  capable  of  reproducing  human  behavior  under 
a  wide  range  of  conditions. 

Keywords — Internal  models,  motor  learning,  adaptive  inverse 
control,  motor  control  modelling,  tracking  task 

I.  Introduction 

IN  a  recent  experimental  study  [1],  human  open-loop 
tracking  data  was  obtained.  This  study  revealed 
unexpected  open-loop  response  characteristics  (obtained  by 
response  cursor  blanking)  which  could  not  be  explained  by 
existing  adaptive  control  systems  human  motor  control 
models  such  as  Feedback  Error  Learning  (FEL)  [2-4]  or 
Adaptive  Model  (AMT)  [5,  6].  A  new  human  motor  control 
model,  developed  as  a  nonlinear  generalization  of  AMT, 
exhibits  open-loop  behaviour  differing  from  these  models. 
This  generalization,  called  nonlinear  AMT  (nAMT),  was 
used  to  simulate  the  experimental  open-loop  tracking  data 
with  the  aim  of  explaining  the  open-loop  spectral 
characteristics. 
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Fig.  1 .  nAMT  model  structure  used  in  simulations. 
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II.  Primary  Features  of  Experimental  Results 

The  tracking  study  reported  in  [1]  produced  response 
trajectories  exhibiting  unusual  characteristics,  particularly 
when  the  subjects  were  deprived  of  response  feedback.  The 
primary  features  of  the  spectral  characteristics  of  the  response 
trajectories  are  summarized  here  and  represented  graphically 
in  the  results  section  (see  Fig.  3  and  Fig.  5). 

The  open-loop  responses  for  all  controlled  systems 
showed  a  mean  high-pass  gain  with  a  cutoff  frequency  of 
approximately  0.3  Hz.  This  attenuation  of  low  frequencies 
was  unexpected.  The  mean  open-loop  phase  responses 
exhibited  a  phase  lead  relative  to  the  mean  closed-loop  phase. 
This  phase  lead  effect  was  much  stronger  for  the  nonlinear 
static  system,  which  exhibited  an  open-loop  phase  lead 
peaking  at  15-20°.  The  dynamic  linear  system  also  showed  a 
mean  phase  lead  from  0-3  Hz  which  was  absent  in  closed- 
loop.  The  open-loop  results  also  exhibited  a  characteristic 
low-frequency  drop  in  mean  coherence. 

The  open-loop  results  do  not  suggest  that  the  human 
subjects  are  attempting  to  reach  a  transfer  function  of  unity, 
representing  accurate  tracking  (as  has  been  observed  for 
closed-loop  tracking).  In  fact,  a  quite  different  relationship 
between  target  and  response  appears  to  have  emerged. 

In  contrast,  the  closed-loop  gain  was  close  to  unity  (as 
expected  for  the  human  operator  after  extensive  training) 
across  the  target  bandwidth.  No  attenuation  of  low 
frequencies  was  evident.  The  mean  phase  responses  were 
close  to  zero,  except  for  the  dynamic  linear  system  which 
exhibited  uncompensated  phase  lag.  The  mean  closed-loop 
coherence  was  very  close  to  unity  for  all  three  systems, 
indicating  relatively  little  noise  or  nonlinearity. 
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III.  Methodology 

A.  Unblanked  Simulation  Structure 

The  model  structure  shown  in  Fig.  1  was  used  in  the 
closed-loop  training  phase  of  all  simulations.  The  inverse 
model  is  formed  by  placing  the  forward  model  into  a  high- 
gain  internal-feedback  loop  [7-9].  In  this  model,  the 
musculoskeletal  system  is  considered  to  be  perfectly 
compensated  for  (since  the  experimental  results  were 
recorded  after  substantial  zero-order  practice)  and  is  therefore 
absent  from  the  diagram.  The  target  signal  T(t)  and  the 
external  system  P  were  taken  directly  from  the  experimental 
study.  The  remaining  model  parameters  were  initialized  with 
typical  parameters  for  a  normal  individual  [10]. 


response  with  an  appropriate  closed-loop  learning  time 
constant  (compared  with  experimental  results  from  human 
subjects)  and  these  results  are  therefore  reported. 

The  internal  feedback  loop  gain  K,  used  to  invert  the 
forward  model,  was  defined  as 

K(e)  =  Pe  +  I  \e.dt  +  D-, 

J  dt 

where  e  is  the  feedback  loop  error. 

The  internal  feedback  loop-gain  parameters  previously 
found  to  produce  good  results  for  a  zero-order  system  were 
used  for  these  runs  (P  =  0.5,  £>  =  1,  7=0.05).  Trial  runs 
with  various  other  feedback-loop  gains  produced  no  clear 
improvements,  and  it  was  very  desirable  to  maintain  similar 
settings  between  external  systems. 


B.  Blanked  Simulation  Structure 

Response  cursor  blanking  was  simulated  by  removing 
response  feedback  R(t)  from  the  model  shown  in  Fig.  1.  It 
is  unclear  exactly  how  the  nervous  system  compensates  for 
the  loss  of  this  feedback  signal  and  how  best  to  represent  this 
compensation  in  the  model. 

The  most  efficient  course  of  action  would  likely  involve 
replacing  R(t)  with  the  most  accurate  internal  estimate  of  that 
signal.  R(t)  can  be  replaced  at  two  places  in  Fig.  1: 
immediately  after  sensory  feedback  reaches  brain  R(t  -  /?) , 
or  immediately  prior  to  reaching  the  response  planner  (called 
the  Optimum  Trajectory  Generator,  or  OTG  in  AMT)  at 
R(t  +  a).  Investigation  revealed  that  replacing  R(t  +  a) 
with  R*{t+a)  produced  the  results  most  consistent  with 
experimental  data  (see  Fig.  2),  which  is  equivalent  to 
suspending  corrective  action  during  open-loop  control. 

IV.  Simulation  Procedure 

A.  Dynamic  Linear  External  System  Simulation 

Simulation  of  the  experimental  runs  began  with  the 
dynamic  linear  external  system  used  by  Group  A  in  the 
experiment  (3rd  order  Chebyshev  Type  I  low-pass  filter  with 
cutoff  frequency  of  3  Hz).  Fifteen  runs  were  simulated  in 
accordance  with  the  15  experimental  runs  performed  by  the 
human  subjects.  The  model  was  initialized  with  several 
different  adaptation  coefficients  p  so  that  an  appropriate 
learning  rate  could  be  determined,  p  =  0.001  produced  a 
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R*(t  +  a)  when  R(t)  is  unavailable 
R(t  +  a)  otherwise 


Fig.  2.  Open-loop  Structure  1 .  Approach  based  on  suspending  corrective 

action. 


B.  Static  Nonlinear  External  System  Simulation 


For  consistency,  exactly  the  same  experimental  procedure 
and  model  parameters  used  for  the  dynamic  linear  system 
were  used  in  this  simulation.  The  external  system  was  as 
used  for  Group  B  in  the  experiment  defined  by 
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where  6  is  the  steering  wheel  input  angle  (in  deg). 


C.  Analysis 

A  frill  quasi-linear  analysis  using  the  correlation  method  [11] 
was  performed,  in  which  the  transfer  function  and  coherence 
functions  were  calculated.  The  remnant  power  function  was 
represented  by  calculating  the  coherence  function  (which 
specifies  essentially  the  same  information).  Spectral 
estimates  were  calculated  at  6  frequencies  within  the  target 
bandwidth  of  0.6  Hz,  providing  a  frequency  resolution  of  0.1 
Hz. 


V.  Results 

A.  Dynamic  Linear  System 

The  transfer  function  and  coherence  plots  for  the  final  run 
(run  15)  are  shown  in  Fig.  3.  The  closed-loop  results  show  a 
phase  lag  and  a  gain  above  0.9  at  all  frequencies.  The 
coherence  is  also  very  high  in  closed-loop.  These  superior 
closed-loop  results  are  due  to  the  action  of  the  feedback 
controller.  The  experimental  results  exhibit  less  coherence 
and  more  variance  than  the  simulation,  which  was  expected 
because  noise  was  not  included  in  the  simulation. 

The  open-loop  results  show  the  same  characteristic  drop 
in  gain  and  coherence  below  0.3  Hz  as  was  observed  in  the 
experimental  results.  The  open-loop  results  also  show  a 
phase  lead  consistent  with  the  results  of  several  individual 
subjects,  though  this  effect  is  obscured  in  the  mean. 

By  the  final  run,  the  simulated  open-loop  trajectory 
exhibits  characteristics  visually  similar  to  the  mean  response 
trajectories  of  the  experimental  subjects  (Fig.  4).  The 
simulated  trajectory  remains  within  one  standard  deviation  of 
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Fig.  3.  Simulated  a)  transfer  function  and  b)  coherence  for  dynamic  linear 
external  system  after  1 5  runs.  Bars  indicate  standard  deviation.  Mean 
experimental  results  (final  run)  shown  with  solid  lines.  Bars  indicate  sd. 
Simulated  results  indicated  with  circles. 


Fig.  4.  Simulated  open-loop  response  after  15  runs  (dynamic  linear  system). 
Dashed  lines  indicate  experimental  sd. 

the  mean  experimental  response.  The  simulated  trajectory 
could  well  represent  a  typical  human  response. 

B.  Static  Nonlinear  System 

The  transfer  function  and  coherence  plots  for  the  final  run 
are  shown  in  Fig.  5.  In  closed-loop  the  simulated  gain  below 
0.5  Hz  was  higher  than  observed  experimentally.  The  phase 
response  was  very  close  to  the  experimental  mean,  and  the 
simulated  coherence  exhibited  a  characteristic  drop  above  0.4 
Hz  as  seen  experimentally. 


Fig.  5.  Simulated  a)  transfer  function  and  b)  coherence  for  static  nonlinear 
external  system  after  15  runs.  Bars  indicate  standard  deviation.  Mean 
experimental  results  (final  run)  shown  with  solid  lines.  Bars  indicate  sd. 
Simulated  results  indicated  with  circles. 


Fig.  6.  Simulated  open-loop  response  in  relation  to  the  target  after  15  runs 

(static  nonlinear  system).  Dashed  lines  indicate  experimental  sd. 

The  open-loop  results  show  the  same  drop  in  gain  and 
coherence  below  0.3  Hz  that  was  characteristic  of  the  mean 
experimental  results.  The  gain  is  lower  than  the  experimental 
mean  across  the  target  bandwidth  but  remains  within  one 
standard  deviation  of  the  mean  except  at  very  low 
frequencies..  The  simulated  drop  in  coherence  is  stronger 
than  that  observed  in  the  mean  human  data,  with  a 
particularly  clear  disparity  at  very  low  frequency  (0  Hz  in 
Fig.  5).  The  simulated  open-loop  results  show  a  strong  phase 
lead  for  frequencies  in  the  lower  half  of  the  target  bandwidth. 
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The  experimental  mean  exhibits  an  approximately  linear 
phase  lead  to  reach  20°  at  0.6  Hz.  In  contrast,  in  the 
simulated  phase  response,  a  phase  lead  at  low  frequencies 
reverses  at  higher  frequencies,  resulting  in  a  slight  phase  lag 
from  0.5-0. 6  Hz. 

Fig.  6  shows  the  simulated  open-loop  response  trajectory 
for  run  15.  The  response  shows  very  similar  characteristics 
to  the  unusual  response  trajectories  observed  in  the 
experiment.  The  simulated  trajectory  again  remains  within 
one  standard  deviation  of  the  mean  experimental  response 

VI.  Discussion 

The  nAMT  computational  model  succeeded  in  generating 
responses  which  reproduced  many  of  the  principle 
characteristics  of  the  response  trajectories  obtained  during  the 
experimental  study.  Importantly,  these  results  were  obtained 
using  a  single  set  of  parameters  across  the  wide  range  of 
conditions  studied  in  the  experiment  (i.e.,  both  closed-  and 
open-loop  control  and  several  different  external  systems).  As 
hoped,  the  nAMT  model  was  able  to  suggest  a  possible  cause 
for  the  unusual  open-loop  trajectories. 

It  proved  possible  to  generate  a  high-pass  gain  with  a 
phase  lead  in  the  absence  of  response  feedback  while 
retaining  acceptably  accurate  performance  in  closed-loop  (see 
Fig.  3  and  Fig.  5).  The  ability  of  the  model  to  achieve  these 
results  arose  from  the  inversion  method  employed  in  nAMT, 
the  internal  feedback  loop,  which  argues  in  favor  of  the 
existence  of  similar  circuitry  in  the  brain  [9].  The  internal 
loop-gain  found  to  optimally  reproduce  the  experimental 
results  was  found  to  be  surprisingly  low  ( P  =  0.5,  D  =  1,  1  = 
0.05),  suggesting  a  dominant  role  for  feedback  control  at  this 
target  bandwidth  (0.6  Hz).  It  is  notable  that  both  FEL  and 
AMT  are  incapable  of  reproducing  this  disparity  between 
closed-  and  open-loop  results  without  modification  to  their 
structures. 

It  is  usually  suggested  that  during  learning  the  inverse 
model  becomes  increasingly  accurate  until  final  convergence 
is  achieved.  In  a  combined  adaptive  control  structure  the 
open- loop  response  trajectory  would  consequently  become 
increasingly  similar  to  the  target.  There  is  no  obvious  reason 
why  only  the  high  frequencies  should  be  learned  accurately, 
as  appears  to  be  the  case  in  the  experimental  results.  This  is 
particularly  true  for  a  target  signal  with  an  approximately  flat 
frequency  response  like  that  used  in  this  study. 

In  simulation,  both  AMT  and  FEL  behave  as  predicted: 
the  inverse  model  becomes  increasingly  accurate, 
particularly  at  low  frequencies.  Indeed,  to  prevent  an 
accurate  inverse  model  forming  at  low  frequencies  an 
artificial  filter  needs  to  be  added  to  deliberately  disrupt  the 
model.  While  such  low  frequency  disruption  can  be 
compensated  for  by  the  closed-loop  controller,  and  hence 
would  not  affect  normal  performance,  it  is  difficult  to  suggest 
why  the  brain  would  disrupt  the  inverse  in  this  manner. 

In  nAMT,  using  a  low  internal  feedback  loop-gain 
maximizes  the  stability  of  the  loop  but  also  essentially  filters 
the  motor  response.  This  prevents  the  inverse  from  becoming 
accurate  at  all  frequencies.  This  inversion  method  trades 


inverse  model  accuracy  for  stability.  Even  when  the  forward 
model  is  entirely  accurate  it  may  be  necessary  to  keep  the 
loop-gain  low  to  maintain  inverse  loop  stability.  Hence, 
unlike  other  combined  motor  models,  the  nAMT  structure 
could  potentially  finish  learning  with  a  completely  accurate 
forward  model  but,  due  to  a  low  loop-gain,  retain  an 
inaccurate  inverse  indefinitely. 

The  experimentally  observed  behavior  arose  from  the 
nAMT  structure  with  no  major  additions  or  alterations.  This 
is  supportive  of  the  claim  that  an  internal  feedback  loop  is 
used  for  the  inversion  of  external  systems  in  the  human  brain 
[9].  Paradoxically,  the  standard  AMT  model,  FEL  and,  to 
our  knowledge,  all  other  control-systems-type  motor  control 
models  are  incapable  of  reproducing  these  results  due  to  the 
accuracy  of  the  inversion  techniques  they  employ. 
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